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- The MAILING DATE of this communication appears on the cover sttoet with the correspondence address — 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time maybe available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 1 33). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )H Responsive to communication(s) filed on 8/29/2001 . 
2a)D This action is FINAL. 2b)IE This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) [x] Claim(s) 1-10 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) M Claim(s) 1-6 and 8-10 is/are rejected. 

7) M Claim(s) 7 is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement 

Application Papers 

9) D The specification is objected to by the Examiner. 

10) 13 The drawing(s) filed on 29 August 2001 is/are: a)D accepted or b)[X] objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

1 1) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12) 13 Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 
a)H All b)D Some * c)D None of: 

1 -13 Certified copies of the priority documents have been received . 

2. D Certified copies of the priority documents have been received in Application No. . 

3. Q Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

Attachment(s) 

1) E3 Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-413) 

2) U Notice of Drafts person's Patent Drawing Review (PTO-948) Paper No(s)/Mail Date. . 

3) I3 Information Disclosure Statement(s) (PTO-1449 or PTO/SB/08) 5) □ Notice of Informal Patent Application (PTO-1 52) 

Paper No(s)/Mail Date 11/20/2001 . 6) □ Other: . 

U.S. Patent and Trademark Office ~ " ~ ~ ~ 

PTOL-326 (Rev. 1-04) Office Action Summary Part of Paper No./Mail Date 20040819 
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DETAILED ACTION 
Drawings 

1 . Figure 1 should be designated by a legend such as —Prior Art— because only that which is 
old is illustrated. See MPEP § 608.02(g). 

2. The drawings are objected to as failing to comply with 37 CFR 1.84(p)(4) because 
reference character "73" of Fig. 7 has been used to designate both a computer and program 
storage. Corrected drawing sheets in compliance with 37 CFR 1. 121(d) are required in reply to 
the Office action to avoid abandonment of the application. 

Any amended replacement drawing sheet should include all of the figures appearing on 
the immediate prior version of the sheet, even if only one figure is being amended. The 
replacement sheet(s) should be labeled "Replacement Sheet" in the page header (as per 37 CFR 
1 .84(c)) so as not to obstruct any portion of the drawing figures. If the examiner does not accept 
the changes, the applicant will be notified and informed of any required corrective action in the 
next Office action. 

The objection to the drawings will not be held in abeyance. 

Claim Rejections - 35 USC §101 



3. 35 U.S.C. 101 reads as follows: 
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Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or 
any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and 
requirements of this title. 

4. Claim 10 is rejected under 35 U.S.C. 101 because the claimed invention is directed to 
non-statutory subject matter. As per the MPEP 2106: 

Since a computer program is merely a set of instructions capable of being executed by a computer, the 
computer program itself is not a process and Office Personnel should treat a claim for a computer program, 
without the computer-readable medium needed to realize the computer program's functionality, as 
nonstatutory functional descriptive material. 

Thus, since Claim 10 recites a computer program without a computer readable medium, it 
is considered to be non-statutory subject matter. 

Claim Rejections - 35 USC §102 

5. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by another filed 
in the United States before the invention by the applicant for patent or (2) a patent granted on an application for 
patent by another filed in the United States before the invention by the applicant for patent, except that an 
international application filed under the treaty defined in section 351(a) shall have the effects for purposes of this 
subsection of an application filed in the United States only if the international application designated the United 
States and was published under Article 21(2) of such treaty in the English language. 

6. Claims 1, 4, 6, 8, and 9 are rejected under 35 U.S.C. 102(e) as being anticipated by 
Bohacek et al (U.S. Patent: 6,411,687). 

With respect to Claims 1 and 9, Bohacek discloses: 
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A voice information input part for inputting user's voice information from a user terminal 
(user phone call to an interactive voice response system which would inherently require a 
telephone or similar communication device, Col. 3, Lines 60-65); 

A voice recognition part for conducting voice recognition with respect to the voice 
information, and analyzing contents of the voice information (speech recognizer used to collect 
information such as account number or billing data, Col. 3, Line 48- Col 4, Line 8); 

A voice information mediation part for controlling a transmission path of the voice 
information in accordance with the contents of the voice information (detecting user mood based 
upon speech samples to transfer an IVR system user to an operator for assistance, Col. 4 t Lines 
40-48, and Col 3, Lines 17-30); 

An interaction engine for extracting contents of a response corresponding to the voice 
information by referring to a knowledge database, and creating^ a synthesized voice in accordance 
with the contents of a response (interactive voice response unit that prompts a user for 
information, Col 3, Lines 60-65, which would inherently require a memory for storing voice 
prompts); and 

A voice information output part for outputting the synthesized voice, wherein the voice 
information mediation part monitors at all times whether or not the user's interaction is being 
smoothly conducted, and in a case of determining that the user's interaction is not being 
smoothly conducted, allows a third-party user to participate in interaction between the user and 
the interaction engine from another terminal as a helper (requesting user information, Col 3, 
Lines 60-65, and transferring the call of a frustrated IVR system user to an operator, Col 3, 
Lines 17-30, and Col 4, Lines 40-48). 
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With respect to Claim 4, Bohacek recites: 

The voice information mediation part determines whether or not the user's interaction is 
being smoothly conducted based on an evaluation value of average sound quality of the voice 
information input by the user, and in a case where the evaluation value of average sound quality 
exceeds a first threshold value or in a case where the evaluation value of average sound quality is 
below a second threshold value, allows a third-party user to participate in the interaction between 
the user and the interaction engine from another terminal as a helper (transferring a call to an 
operator when an annoyance threshold, which considers speech data such as the pitch of a 
sample, is exceeded, Col 3, Lines 1-30). 

With respect to Claim 6, Bohacek discloses: 

The interaction engine further includes an interaction history information storage part for 
recording interaction history on a user basis, and a helper selection part for selecting the third- 
party user that is considered to be most familiar with the contents of the interaction from the 
interaction history as a helper, and the helper most appropriate for contents of the voice 
information is selected (previous customer data used to route a call to the most appropriate 
operator, Col. 4, Lines 24-39, and Fig. 1). 

With respect to Claim 8, Bohacek recites: 

Interaction history display part for displaying the interaction history stored in the 
interaction history information storage part to a third-party helper user, and a helper instruction 
part for receiving a help instruction from the third-party helper user, wherein when the help 
instruction part receives the help instruction from the third-party helper user, the voice 
information mediation part enables the interaction between the third-party helper user and the 
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user to be conducted, and when a degree of help of the third-party helper user exceeds a 
predetermined threshold value in interaction between the third-party helper user and the user, the 
interaction engine interacts only with the third-party helper user (transferring interaction control 
to an appropriate operator when an annoyance threshold is exceeded, displaying customer data, 
and previous customer data, Col. 3, Lines 17-30, and Col. 4, Lines 30-39), 

Claim Rejections - 35 USC §103 

1. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

8. Claims 2 and 3 are rejected under 35 U.S.C. 103(a) as being unpatentable over Bohacek 

et al in view of Marx et al (U.S. Patent: 6,173,266). 

With respect to Claim 2, Bohacek teaches the interactive voice response system and 

method capable of transferring a caller to an operator upon detecting a predetermined level of 

frustration based on speech data, as applied to Claim 1 . Bohacek does not specifically suggest 

the ability to transfer a call to an operator when a recognition error limit is exceeded, however 

Marx discloses: 

The voice information mediation part determines whether or not the user's interaction is 
being smoothly conducted based on whether or not an accumulation value of the number of 
times at which the contents of a response are not found in the knowledge database exceeds a set 
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limit number, and in a case where the accumulation value exceeds the set limit number, allows a 
third-party user to participate in the interaction between the user and the interaction engine from 
another terminal as a helper (exceeding a predetermined number of recognition attempts to 
initiate a call transfer to a live operator, Col. 10, Lines 7-40). 

Bohacek and Marx are analogous art because they are from a similar field of endeavor in 
interactive voice response systems. Thus, it would have been obvious to a person of ordinary 
skill in the art, at the time of invention, to combine the means of transferring a call to a live 
operator upon exceeding a predetermined number of recognition errors as taught by Marx with 
the interactive voice response system capable of transferring a caller to an operator upon 
detecting a predetermined level of frustration based on speech data as taught by Bohacek to 
provide a further means of assisting an IVR system user in accomplishing a desired call result 
when speech data does not indicate a certain degree of frustration, but repeated recognition errors 
occur. Therefore, it would have been obvious to combine Marx with Bohacek for the benefit of 
providing proper user assistance upon the detection of repeated speech recognition errors. 

With respect to Claim 3, Bohacek teaches the interactive voice response system and 
method capable of transferring a caller to an operator upon detecting a predetermined level of 
frustration based on speech data, as applied to Claim L Bohacek does not specifically suggest 
the ability to transfer a call to an operator when a response reaction time limit is exceeded, 
however, Marx recites: 

The voice information mediation part determines whether or not the user's interaction is 
being smoothly conducted based on an average reaction time from a response of the interaction 
engine to a reaction of the user, and in a case where the average reaction time exceeds a first 
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threshold value or in a case where the average reaction time is below a second threshold value, 
allows a third-party user to participate in the interaction between the user and the interaction 
engine from another terminal as a helper (timeout condition used to initialize a call transfer to an 
operator, Col 8, Lines 20-31, and Col. 9, Lines 52-65). 

Bohacek and Marx are analogous art because they are from a similar field of endeavor in 
interactive voice response systems. Thus, it would have been obvious to a person of ordinary 
skill in the art, at the time of invention, to combine the use of a timeout condition to initialize a 
call transfer to an operator as taught by Marx with the interactive voice response system capable 
of transferring a caller to an operator upon detecting a predetermined level of frustration based 
on speech data as taught by Bohacek in order to provide a further means of assisting an IVR 
system user in accomplishing a desired call result when a lack of response indicates possible user 
confusion. Therefore, it would have been obvious to combine Marx with Bohacek for the benefit 
of providing proper user assistance in response to user confusion indicated by a lack of response. 

9. Claim 5 is rejected under 35 U.S.C. 103(a) as being unpatentable over Bohacek in view 
of Marx, and further in view of Davis et al (U.S. Patent: 5,583,922). 

With respect to Claim 5, Bohacek in view of Marx teaches the interactive voice response 
system and method capable of transferring a caller to an operator based upon a frustration level, 
recognition engine error amount, or a timeout condition, as applied to Claims 1-3. Bohacek in 
view of Marx does not specifically suggest the ability to determine interaction progress through 
interaction time and access amounts in order to initialize parallel input operator/user interaction, 
however Davis recites: 
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The contents of interaction with the user is displayed to the third-party user and the 
contents of interaction is updated by the third-party user, parallel input in which the third-party 
user conducts an input in parallel with the user, to switching in which the third-party user 
directly interacts with the user (user information transferred to an operator and direct 
communication through data or voice, Col 11, Lines 48-58). 

Bohacek, Marx, and Davis are analogous art because they are from a similar field of 
endeavor in interactive voice response systems. Thus, it would have been obvious to a person of 
ordinary skill in the art, at the time of invention, to combine the use of a parallel user/operator 
input as taught by Davis with the interactive voice response system and method capable of 
transferring a caller to an operator based upon a frustration level, recognition engine error 
amount, or a timeout condition as taught by Bohacek in view of Marx in order to provide a 
means for a user to maintain some system control while being aided in information entry by an 
operator to ensure data accuracy in the condition that recognition or timeout errors occur. 
Therefore, it would have been obvious to combine Davis with Bohacek in view of Marx for the 
benefit of providing parallel input user assistance. 

10, Claim 10 is rejected under 35 U.S.C. 103(a) as being unpatentable over Bohacek, 

With respect to Claim 10, Bohacek teaches the interactive voice response system and 
method capable of transferring a caller to an operator upon detecting a predetermined level of 
frustration based on speech data, as applied to Claim 1. Although Bohacek does not specifically 
suggest method storage on a computer readable medium, the examiner takes official notice that it 
would have been obvious to one of ordinary skill in the art, at the time of invention, to store the 
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IVR method taught by Bohacek on a computer readable medium to increase method 
compatibility and usability by providing a means for method use with multiple computer 
systems. 

Allowable Subject Matter 

1 1 . Claim 7 is objected to as being dependent upon a rejected base claim, but would be 
allowable if rewritten in independent form including all of the limitations of the base claim and 
any intervening claims. 

12. The following is a statement of reasons for the indication of allowable subject matter: 
With respect to Claim 7, the prior art of record does not teach nor fairly suggests: the 

ability of an operator to assume complete control of an interaction engine in an interactive voice 
response system when it is detected that only an operator's voice continues for a predetermined 
time period during an error-initiated operator interaction session, in combination with a help 
request notification used for informing an operator of user errors that occur in an interactive 
voice response system session and allowing the operator to voluntarily interact with the user in 
response to such a notification. 

Conclusion 

13. The prior art made of record and not relied upon is considered pertinent to applicants 
disclosure: 
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• Dowden et al (U.S. Patent: 5,163,083)- teaches a means for transferring a caller 
to an operator upon the occurrence of an error condition in an IVR system. 

• Mitchell et al (US. Patent: 5,164,981)- teaches a voice response system that 
transfers a list of interaction data to an operator upon switching to an operator 
assistance mode. 

• Muthusamy et al (U.S. Patent: 5,805, 771)- discloses a method of transferring a 
caller to a most appropriate operator in an IVR application based upon speech 
analysis. 



14. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to James S. Wozniak whose telephone number is (703) 305-8669 
and email is James.Wozniak@uspto.gov. The examiner can normally be reached on Mondays- 
Fridays, 8:30-4:30. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Doris To can be reached at (703) 305-4827. The fax/phone number for the 
Technology Center 2600 where this application is assigned is (703) 872-9306. 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the technology center receptionist whose telephone number-is (703) 306- 
0377. 



James S. Wozniak 
8/23/2004 




